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The world is changing 


original Plan9 


e [imesharing from commodity 

e small 

e statically administered 

e low bandwidth terminals 

e closely coupled file and cpu servers 
e everything is a file/9p everywhere 


e strong security model 
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World today 


Client/server interaction based on HTTP 


Poor, ad-hoc security 


Many layers of gratuitous protocol 


e REST over Websockets over HTTP over T CP 
over IP 


No model for accessing replicated static content 


e Not to mention dynamic content 
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World tomorrow 


Large elastic "clouds" of machines 


Machines and applications are mobile 


Lots of replicated and distributed "content" 


Mobile applications supported by network 
services 
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Modern problems 


authentication and access control of network content and 
services 


e http has no built in auth mechanisms other than basic 
e http not efficient 

e reloading a bunch of stuff over tcp 
e constant mobility 
e "connection" equated to "session" 


e connections just aren't reliable in a modern world 
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Plan 9 solves some 
problems 


e Well authenticated RPC protocol(9p) 


e Name spaces allow location-independent 
access 
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But not all 


e | want to... 


dynamically add xen systems to my NDB. 
e have my auth server proxy auth for me 
€ make my system come to me. 


e data computation etc.. 


keep my state after drawterm crashes. 
e manage large-scale distributed applications. 


© manage dynamic, replicated data. 


Wednesday, October 13, 2010 


Solving these problems 
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Osprey, what 


Authenticated, controlled access to dynamic 
replicated data 


support for cloud applications: remote invocation, 
checkpointing, migration 


good for new multicore processors 
good for hard real-time embedded development 


support for mobility and less than fully reliable 
wireless communication 


Osprey, how 


e microkernel w/library oses on top 
e distributed delegated auth 


© new session based file protocol, good for 
streaming and persistence 


e migration possible 
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microkernel 


e minimal state in the kernel only: 
e fast message passing 
€ address space maintenance 
e interrupt handling 


e process scheduling 
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fast message passing 


inter-core message passing 


page flipping where possible 
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Internals 


address spaces 
segments can be shared different addresses 
processes share segments 


processes that to share address spaces 
migrate as a group 


static void 

uarttask(void *arg) 

( 
char c; 
Uart *uart = arg; 
Task *ut = m->task; 


Msg *msg; 
Select sels[4] = ( 
{.op = MSget, (.g=ut), {.p=&msg}  }, 


{ .op = MSdown, { .s = uart->sem ), {.b=nil} ), 
{.op = MSget,  4.g = uart->gout }, {.b=&c} }, 
{.op = MSlast, 4.gznil), (.b=nil» ), 

y 


int n; 


/* Starts when interrupts are enabled */ 
while(uart->enabled)4 
/* 
* don't receive characters from output queue unless 
* there's room in the fifo: 
*/ 
sels[2].op = uart->sendrdy ? MSget : MSnoop; 
/* wait for an event: */ 
n = select(sels); 
F prevent interrupts, this IS a device driver: */ 
ilock(uart); 
switch(n)( 
default: 
_assert("i8250task: select"); 
case 0: 
F process message */ 
uartmsg(uart, msg); 
case 1: 
/* process an interrupt */ 
uart->phys->interrupt(uart); 
break; 
case 2: 
F print character from output queue */ 
uart->phys->sendc(uart, c); 
break; 


} 


iunlock(uart); 


} 

iprint("i8250task exits\n"); 
RAZ DZ  _— eee eee NJEJ + 
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Tip 


similar to 9p but sessions are now independent from 
connections 


e ie.if your connection dies your session stays alive if 
you want it to 


transactions can be grouped "pipelined" 


e ie, file can be opened, read and closed in one round 
trip. 


can be its own transport protocol 


support for isochronous data 
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the more things stay 
the same 


e authentication 
e fids 


e | and R messages 


the more they change 


e string attributes 
© versioning 

e all files are versioned 

e immutable, committed on file close 
e file are leased 


e clients can renegotiate session 
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uart 2.5358 Frepare topen 

devroot  2.553s parse group 

devroot 2.555s session:sid 12345678, tag 4 

devroot 2.558s topen, fid I, nfid 2, name dev/uart/eia0, how r 

devroot 2.564s topen, chan /, fref 0x0/0x0 

devroot 2.567s topen, cloned fid 

devroot 2.567s topen, walk /dev/uart/eia0 

devroot 2.567s topen, dev->walk / to dev 

devroot 2.576s rootwalk / (0/0/0) — dev 

devroot 2.578s devwalk /dev (0x200 0/2/0) 

devroot 2.585s topen, path /dev 

devroot 2.587s topen, dev->walk /dev to uart 

devroot  2.587s rootwalk /dev (0/2/0) > uart 

devroot  2.587s devwalk /dev > uart 

devroot  2.594s devwalk /dev/uart (0x2 2/0/0) 

devroot  2.600s topen, path /dev/uart 

devroot 2.600s topen, dev->walk /dev/uart to eia0 

devroot  2.605s uartwalk /dev/uart 2 eia0 

devroot 2.732s uartdiscover: uarts Oxffffffffc0230a20, uartdir Oxffffffffc0230a68 
devroot 2.737s uartdiscover: 2 entries 

devroot — 2.739s devwalk /dev/uart = eia0 

devroot  2.741s devwalk /dev/uart/eia0 (0x10002 2/0/1) 

devroot 2.741s uartwalk /dev/uart/eia0 == uart 0, COMI (© OxffffffffcO|2c6d0 
devroot  2.741s topen, path /dev/uart/eia0 

devroot 2.750s uartopen /dev/uart/eia0 r 

devroot 2753s uartopen /dev/uart/eia0: COMI 

devroot  2.755s devopen /dev/uart/eia0 r Oxffffffffc0230a68 

devroot 2.757s session: reply 24 bytes 

uart 2.757s Got reply 

uart 2.757s ropen: done 

uart 2.764s Prepare tread 

devroot  2.78ls parse group 

devroot 2.781s session: sid 12345678, tag 5 

devroot 2.789s session: read: preset rep->data OxffffffffcO22d454, 8204 
devroot  2.794s uartread 

devroot  2.794s uartread 256 characters attribute * 

devroot  2.799s uartread wait for response 

Uarttask — 2.80|s uartmsg get attr * 

devroot  2.803s session: reply 77 bytes 

uart 2.805s Got reply 

rread attr=*: 65 bytes: 4 *bcdefhiklmprs x framing overruns berr cts dsr dcd' 
uart 2.805s Prepare tread 

devroot  2.828s parse group 

devroot 2.830s session: sid 12345678, tag 6 

devroot 2.837s session: read: preset rep->data OxffffffffcO22d454, 8204 
devroot 2.837s uartread 

devroot 2.837s uartread 256 characters attribute b 

Uarttask  2.846s uartmsg get attr b 

devroot 2.837s uartread wait for response 

devroot  2.896s session: reply 16 bytes 

uart 2.896s Got reply 

rread attr=b: 4 bytes: '9600' 

10 seconds type ahead: Uarttask  2.946s uartmsg set attr d 
aSDASDASD 

pte JINIde Prenare tread oo 
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Applications 


High throughput batch jobs 
e routing, gateways, firewalls 
Cloud OS 

e [hin client OS 

e HPC OS 
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Progress 


e boots 


© real time scheduler 


working on a packet filter 


e TTp devices in the kernel 
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Conclusions 


Plan9 is still che right engineering model 
e fs based way of unifying dist system 


but the current implementation is too 
static 


Osprey gets around these problems by 
using a microkernel with migratable 
processes and a caching filesystem 
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Demo 


duvel (10.12.0.67!67): /amd64/kpc64 
109969+28912+77552=216433 
warp64(OxffffffffcO | 10000) Oxffffffffc0000000 16 


This is Osprey 


Ptr size 64 
2666MHz 2666666667Hz 3749995 
pml4 OxffffffffcO | 03000 
memstart 0x145000 2863104 
level 4 page table for va 0x0000000000000000 at OxffffffffcO | 03000 
pte[511] va Oxffffff8000000000 — pa 0x0000000000 10b000 xi ac sw indirect 
level 3 page table for va Oxffffff8000000000 at OxffffffffcO | Ob000 
pte[511] va OxffffffffcO000000 — pa 0x0000000000 0c000 xi ac sw indirect 
level 2 page table for va Oxffffffffc0000000 at OxffffffffcO | 0c000 
pte[ 0] va Oxffffffffc0000000 — pa 0x0000000000000000 x dac sw size 0x200000 
pte[ |] va Oxffffffffc0200000 — pa 0x0000000000200000 x dac sw size 0x200000 
pte[127] va Oxffffffffcfe00000 — pa 0x000000000010d000 xi c sw indirect 
level | page table for va Oxffffffffcfe00000 at OxffffffffcO | Od000 
ELCR: 0E00 
pit0: hz 1193182 max 50000000000000 min 100000000000 mul 5120 58207 
Enable asynchronous printing 
dumptask: task Acedia at OxffffffffcO | 86000 
stack: OxffffffffcO | 88000-0xffffffffcO 188000 
pi pc: 0x0, sp: 0x0 
n messages received: 0 
messages in gueue: 0 
gp periods: 20000409 
on 3.766s best-effort time used 
dumptask: task Uarttask at OxfffffffícO | 88000 
stack: OxffffffffcO | 83000-OxffffffffcO | 81000 
g pc: OxffffffffcO | 23a |f, sp: OxffffffffcO | 89ea8 
:state Ready, scheduler patientia, events 0x0, flags 0x0 
messages received: 0 
messages in gueue: 0 
10 periods: 10 
00 164.538us best-effort time used 
dumptask: task Ira at OxffffffffcO | 81000 
00 stack: OxffffffffcO | 8c000-0xffrfffffcO | 8c000 
pc: OxffffffffcO | 234de, sp: OxfffffffícO | 8be78 
state Waitsem, scheduler patientia, events 0x0, flags 0x0 
00 messages received: 0 
p messages in queue: 0 
in periods: I 
419.999ns best-effort time used 
gdumptask: task pingpong at OxffffffffcO | 8c000 
stack: OxffffffffcO | 8e000-OxffffffffcO | 8e000 
po pc: Oxffffffffc0 | | eace, sp: OxfffffffícO 18dba0 
state Running, scheduler patientia, events 0x0, flags 0x0 
ng messages received: 10000001 
messages in queue: 0 
s periods: 10000228 
i 3.480s best-effort time used 
dumptask: task Secondo at OxffffffffcO | 8e000 
stack: OxffffffffcO | 90000-0xffffffffcO | 90000 
n pc: Oxffffffffc0 | 234de, sp: OxfffffffícO | 8fe90 
state Waitsem, scheduler patientia, events 0x0, flags 0x0 
5 messages received: | 0000000 
messages in queue: 0 
. periods: 10000196 
40 1.954s best-effort time used 
20000456 context switches 
6 60001910 calls to now() 
s 1105 timer interrupts 


This was Osprey 


msgsend+msgrecv+context switch: 270.309ns 
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duvel (10.12.0.67!67):/386/kpc32 
10801 1+24180+40752=172943 
entry: Oxf0 | 10000 
warp9(0x | 10000) || 


This is Osprey 


pointer size 32 
999MHz 999562553Hz | .000ns 
ELCR: OEAO 
pitO: hz 1193182 max 50000000000000 min 100000000000 mul 5120 58207 
Enable asynchronous printing 
Acedia |.374s 18250attr 20 
uart |.446s Trying i8250uart COMI 
uart 1.536s Trying i8250uart COM2 
uaUarttask  1.628s uartmsg get attr * 
rUarttask 1.714s uartmsg get attr b 
Uarttask  1.798s i8250attr 2 
tuart 1.869s i8250attr 8 
Uarttask 1.938s uartmsg set attr d 
Uarttask  2.020s i8250attr 8 
|.445s Starting 
uart |.446s Allocate memory 
uart |.446s uartdiscover 
uart l.625s uartdiscover: uarts 0xf023d5a0, uartdir 0xf023d5e8 
uart |.626s uartdiscover: 2 entries 
uart 1.627s uartopen /dev/uart/eia0: COM | 
rread attr=*: 65 bytes: 4 * bcdefhiklIm pr s x framing overruns berr cts dsr dcd' 
rread attr=b: 4 bytes: '9600' 
10 seconds type ahead: 
huart 12.974s i8250attr 8 
rrUarttask  |3.043s uartmsg set attr d 
Uarttask  |3.127s i8250attr 8 
ead: | bytes: 'h' 


This was Osprey 
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